1. Technical Field
The present disclosure is related to a method and device for detecting a touch between at least part of a first object and at least part of a second object, wherein the at least part of the first object has a different temperature than the at least part of the second object. The disclosure is also related to a computer program product comprising software code sections which are adapted to perform such method.
2. Background Information
A natural way for humans to interact with (real) objects is to touch them with their hands. For example, in current Augmented Reality (AR) applications, interaction with real and virtual objects usually involves the user's hand and a screen displaying an image of the real object, instead of interacting with real objects directly. Such screens that allow detecting and localizing touches on their surface are commonly known as touch screens and are nowadays common part of, e.g., smartphones and tablet computers. A current trend is that displays for AR are becoming smaller and/or they move closer to the retina of the user's eye. This is for example the case for head-mounted displays, and makes using touch screens difficult or even infeasible.
One possible solution in this case is to use occlusion-based interaction methods, such as described in references PCT Patent Publication No. WO 2013/016104 A1 and “Occlusion Based Interaction Methods for Tangible Augmented Reality Environments”, Lee et al., VRCAI '04 Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industry, pages 419-426, 2004; (hereinafter “Lee”). In these methods, an interaction event is triggered if a certain area of a real object is occluded from the viewpoint of a camera. Note that the camera's pose (i.e. position and orientation) with respect to the real object needs to be known to being able to identify such occlusions. This pose can either be determined once offline or continuously during runtime which allows for motion of the real object and/or the camera. Such occlusion-based interaction is, for example, part of the Vuforia SDK under the name “virtual buttons”. Occlusion-based virtual buttons have the following shortcomings: they cannot distinguish if an object (e.g. fingertip) actually touches the virtual button or if it only occludes it and they cannot distinguish if the occlusion (or touch) is caused by a finger(tip) on purpose or by any other object (e.g. a sleeve hem) by accident.
The following common approaches exist to detect a touch between at least part of a human body and an object. The most common approach is to physically equip the object or the human body (e.g. fingertip) with a sensor capable of sensing touch. This could be anything from a simple mechanical switch to a touch-pad or touch screen. It could, for example, also be based on electrical voltage applied to the body and closing a circuit when touching a real object. The limitation of such kinds of approaches is that they require modifications of the object or the human body.
A touch can also be detected if the pose of the part of the body, e.g. the hand, is known relative to the object. There are many approaches aiming at tracking the pose of a finger or a hand. These can be based on one or more cameras that sense visible light and/or depth, e.g. using a time-of-flight camera, or active stereo based on infrared structured light. There are also approaches that equip a user's hand with sensors, e.g. inertial sensors, to sense the pose of the hand. The limitation of all these approaches is that the determined pose of a hand or finger is too inaccurate to reliably tell if a fingertip touches a real object or if it is only very close to it, e.g. 2 mm apart from the object.
Other approaches, as described in PCT Patent Publication No. WO 2013/016104 A1 and Lee, such as the virtual buttons in the Vuforia SDK mentioned above, do not aim at detecting a touch but at detecting an occlusion resulting in many limitations. Virtual buttons need to have certain size that allows to robustly identify if the button is occluded or not. It would, for example, not be feasible to subdivide an A4 sheet of paper into a grid of 297×210 virtual buttons with a size of 1×1 mm each. This limits the application of virtual buttons to tasks that do not require precise and continuous positional input, but only discrete button triggers. Furthermore, these virtual buttons need to have a visual appearance different from that of a finger, so that occlusions can be identified. The fact that these methods detect occlusions instead of touches results in another severe limitation, which will be discussed in the following at the example of a number pad. With virtual buttons it is impossible to trigger a button, e.g. No. 5 out of an array of adjacent buttons on a number pad, without triggering any other button before, because the button No. 5 cannot be reached without occluding any of the surrounding buttons. This puts heavy constraints on the layout of virtual buttons. Furthermore, while touching or occluding the button No. 5 on a number pad, the hand will additionally occlude other buttons at the same time. In Lee, the authors propose to solve this issue by only considering the top-left button in case multiple buttons are occluded, but this is a very heuristic and unreliable method.
In PCT Patent Publication No. WO 2012/039836 A1, a blow tracking user interface system and method is described, wherein embodiments thereof are directed to user interfaces for control of computer systems, and more specifically to user interfaces that track the blowing of a user's breath to provide control input to a computer program. This blow tracking is done based on thermal infrared imaging.
A user interface system and method using thermal imaging is described in PCT Patent Publication No. WO 2012/040114 A1. It describes a user interface based on obtaining one or more thermal infrared images of one or more objects with one or more thermographic cameras, analyzing the thermal infrared images, identifying characteristics of the objects from the thermal infrared images and using the characteristics as a control input in the computer program. The objects are users and the characteristics are vital signs.
U.S. Patent Publication No. US 2011/0050643 A1 discloses a passive infrared sensing user interface and device using the same. A device includes a passive infrared sensor for user interface. When a user places his finger over the infrared sensor, the sensor generates a digital signal indicating the time, position, or movement of the user finger over the sensor. The user finger does not need to touch or press the infrared sensor, but it can be used for touchless user interfaces.